Deep learning has achieved good results in human action recognition, but it still needs to make full use of video human appearance information and motion information. To recognize human actions by using spatial information and temporal information in video, a video human action recognition model based on spatio-temporal two-stream was proposed. Two convolutional neural networks were used to extract spatial and temporal features of video sequences respectively in the proposed model, and then the two neural networks were merged to extract the middle spatio-temporal features, finally the video human action recognition was completed by inputting the extracted features into a 3D convolutional neural network. The video human action recognition experiments were carried out on the data set UCF101 and HMDB51. Experimental results show that the proposed 3D convolutional neural network model based on the spatio-temporal two-stream can effectively recognize the video human actions.